DEEP-RHYTHM FOR TEMPO ESTIMATION AND RHYTHM PATTERN RECOGNITION

International audienceIt has been shown that the harmonic series at the tempo frequency of the onset-strength-function of an audio signal accurately describes its rhythm pattern and can be used to perform tempo or rhythm pattern estimation. Recently, in the case of multi-pitch estimation, the depth of the input layer of a convolutional network has been used to represent the harmonic series of pitch candidates. We use a similar idea here to represent the harmonic series of tempo candidates. We propose the Harmonic-Constant-Q-Modulation which represents, using a 4D-tensors, the harmonic series of modulation frequencies (considered as tempo frequencies) in several acoustic frequency bands over time. This representation is used as input to a convolutional network which is trained to estimate tempo or rhythm pattern classes. Using a large number of datasets, we evaluate the performance of our approach and compare it with previous approaches. We show that it slightly increases Accuracy-1 for tempo estimation but not the average-mean-Recall for rhythm pattern recognition

Extending Deep Rhythm for Tempo and Genre Estimation Using Complex Convolutions, Multitask Learning and Multi-input Network

Tempo and genre are two inter-leaved aspects of music, genres are often associated to rhythm patterns which are played in specific tempo ranges. In this paper, we focus on the Deep Rhythm system based on a harmonic representation of rhythm used as an input to a convolutional neural network. To consider the relationships between frequency bands, we process complex-valued inputs through complex-convolutions. We also study the joint estimation of tempo/genre using a multitask learning approach. Finally, we study the addition of a second input convolutional branch to the system applied to a mel-spectrogram input dedicated to the timbre. This multi-input approach allows to improve the performances for tempo and genre estimation

Pre-Training Strategies Using Contrastive Learning and Playlist Information for Music Classification and Similarity

In this work, we investigate an approach that relies on contrastive learning and music metadata as a weak source of supervision to train music representation models. Recent studies show that contrastive learning can be used with editorial metadata (e.g., artist or album name) to learn audio representations that are useful for different classification tasks. In this paper, we extend this idea to using playlist data as a source of music similarity information and investigate three approaches to generate anchor and positive track pairs. We evaluate these approaches by fine-tuning the pre-trained models for music multi-label classification tasks (genre, mood, and instrument tagging) and music similarity. We find that creating anchor and positive track pairs by relying on co-occurrences in playlists provides better music similarity and competitive classification results compared to choosing tracks from the same artist as in previous works. Additionally, our best pre-training approach based on playlists provides superior classification performance for most datasets.Comment: Accepted at the 2023 International Conference on Acoustics, Speech, and Signal Processing (ICASSP'23

Vers une estimation globale du tempo et une classification des genres orientée rythme basée sur les caractéristiques harmoniques du rythme

Automatic detection of the rhythmic structure within music is one of the challenges of the "Music Information Retrieval" research area. The advent of technology dedicated to the arts has allowed the emergence of new musical trends generally described by the term "Electronic/Dance Music" (EDM) which encompasses a plethora of sub-genres. This type of music often dedicated to dance is characterized by its rhythmic structure. We propose a rhythmic analysis of what defines certain musical genres including those of EDM. To do so, we want to perform an automatic global tempo estimation task and a genre classification task based on rhythm. Tempo and genre are two intertwined aspects since genres are often associated with rhythmic patterns that are played in specific tempo ranges. Some so-called "handcrafted" tempo estimation systems have been shown to be effective based on the extraction of rhythm-related characteristics. Recently, with the appearance of annotated databases, so-called "data-driven" systems and deep learning approaches have shown progress in the automatic estimation of these tasks. In this thesis, we propose methods at the crossroads between " handcrafted " and " data-driven " systems. The development of a new representation of rhythm combined with deep learning by convolutional neural network is at the basis of all our work. We present in detail our Deep Rhythm method in this thesis and we also present several extensions based on musical intuitions that allow us to improve our results.La détection automatique de la structure rythmique au sein de la musique est l'un des défis du domaine de recherche "Music Information Retrieval". L'avènement de la technologie dédiées aux arts a permis l'émergence de nouvelles tendances musicales généralement décrites par le terme d'"Electronic/Dance Music" (EDM) qui englobe une pléthore de sous-genres. Ce type de musique souvent dédiée à la danse se caractérise par sa structure rythmique. Nous proposons une analyse rythmique de ce qui définit certains genres musicaux dont ceux de l'EDM. Pour ce faire, nous souhaitons réaliser une tâche d'estimation automatique du tempo global et une tâche de classification des genres axée sur le rythme. Le tempo et le genre sont deux aspects entremêlés puisque les genres sont souvent associés à des motifs rythmiques qui sont joués dans des plages de tempo spécifiques. Certains systèmes d'estimation du tempo dit "handcrafted" ont montré leur efficacité en se basant sur l'extraction de caractéristiques liées au rythme. Récemment, avec l'apparition de base de données annotées, les systèmes dit "data-driven" et les approches d'apprentissage profond ont montré des progrès dans l'estimation automatique de ces tâches.Dans cette thèse, nous proposons des méthodes à la croisée des chemins entre les systèmes "handcrafted" et "data-driven". Le développement d'une nouvelle représentation du rythme combiné à un apprentissage profond par réseau de neurone convolutif est à la base de tous nos travaux. Nous présentons en détails notre méthode dites Deep Rhythm dans cette thèse et nous présentons également plusieurs extensions basées sur des intuitions musicales qui nous permettent d'améliorer nos résultats

DEEP-RHYTHM FOR TEMPO ESTIMATION AND RHYTHM PATTERN RECOGNITION

International audienceIt has been shown that the harmonic series at the tempo frequency of the onset-strength-function of an audio signal accurately describes its rhythm pattern and can be used to perform tempo or rhythm pattern estimation. Recently, in the case of multi-pitch estimation, the depth of the input layer of a convolutional network has been used to represent the harmonic series of pitch candidates. We use a similar idea here to represent the harmonic series of tempo candidates. We propose the Harmonic-Constant-Q-Modulation which represents, using a 4D-tensors, the harmonic series of modulation frequencies (considered as tempo frequencies) in several acoustic frequency bands over time. This representation is used as input to a convolutional network which is trained to estimate tempo or rhythm pattern classes. Using a large number of datasets, we evaluate the performance of our approach and compare it with previous approaches. We show that it slightly increases Accuracy-1 for tempo estimation but not the average-mean-Recall for rhythm pattern recognition

Pre-Training Strategies Using Contrastive Learning and Playlist Information for Music Classification and Similarity

This work has been accepted at the 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023), at Rhodes, Greece. June 4-10, 2023.In this work, we investigate an approach that relies on contrastive learning and music metadata as a weak source of supervision to train music representation models. Recent studies show that contrastive learning can be used with editorial metadata (e.g., artist or album name) to learn audio representations that are useful for different classification tasks. In this paper, we extend this idea to using playlist data as a source of music similarity information and investigate three approaches to generate anchor and positive track pairs. We evaluate these approaches by fine-tuning the pre-trained models for music multi-label classification tasks (genre, mood, and instrument tagging) and music similarity. We find that creating anchor and positive track pairs by relying on cooccurrences in playlists provides better music similarity and competitive classification results compared to choosing tracks from the same artist as in previous works. Additionally, our best pre-training approach based on playlists provides superior classification performance for most datasets.This research was partially funded by Musical AI - PID2019-111403GBI00/ AEI/10.13039/501100011033 of the Spanish Ministerio de Ciencia, Innovacion y Universidade